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HALF -NORMAL  PLOTS 

FOR  MULTI-LEVEL  FACTORIAL  EXPERIMENTS 

S.  A.  Krane 

C-E-I-R,  Inc.  , Dugway  Field  Office 

1.  INTRODUCTION.  Half-normal  plots  for  the  interpretation  of 
factorial  experiments  have  been  developed  and  popularized  largely  through 
the  work  of  Cuthbert  Daniel  (see  Daniel  Q95^  and[l95^  ).  In  this 
method  the  2^-1  main  effects  and  interactions  are  estimated  from  ob- 
servations on  the  ZP  treatment  combinations.  The  empirical  cumulative 
distribution  of  these  estimates  is  then  graphically  compared  with  a cu- 
mulative distribution  derived  from  a normal  population.  A rationale  for 
this  procedure  is  found  in  the  approximate  normality  of  the  null  distri- 
bution of  the  estimates,  based  upon  normality  of  experimental  errors  or 
upon  the  tendency  embodied  in  the  Central  Limit  Theorem.  According 

to  Daniel,  the  half-normal  plot  permits  the  analyst  to  judge  the  reality 
of  the  largest  main  effects  and  interactions  and  serves  to  indicate  bad 
values,  heteroscedasticity,  dependence  of  variance  on  mean  and  some 
types  of  defective  randomization.  The  object  of  the  present  paper  is  to 
indicate  and  illustrate  possible  applications  of  half-normal  plots  to 
balance  multi-level  factorial  experiments  in  general. 

2.  AN  EXAMPLE.  It  appears  easiest  to  introduce  the  technique  of 
half -normal  plotting  for  balanced  multi-level  factorial  experiments  in  the 
context  of  a particular  example.  For  this  purpose  we  shall  employ 
Example  8.  1 of  Davies  \j954j.  According  to  the  authors  (p.  291.  "the 
data  . . . are  taken  from  the  results  of  an  investigation  into  the  effects 
on  the  physical  properties  of  vulcanized  rubber  of  varying  a number  of 
factors,  the  property  recorded  being  the  wear  resistance  of  the  samples, 
and  the  factors  being: 

A five  qualities  of  filler 

B three  methods  of  pretreatment  of  the  rubber 

C f our  qualities  of  the  raw  rubber  ..." 

The  data  are  reproduced  in  Table  1.  From  the  data,  the  author  develops 
the  usual  analysis  of  variance  as  shown  in  Table  2.  The  interpretation 
{Davies  [l954,  p.  29^  ) notes  the  significance  of  all  main  effects  and  two- 
factor  interactions  when  tested  against  the  three -factor  interaction  as 


error. 


Table  1.  (Table  8.1  of  Davies  [195^]) 
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Table  2.  (Table  8.16  of  Davies  [1954]) 
ANALYSIS  OF  VARIANCE  OP  TABLE  8.1 
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significant,  that  is  ^ 55^  value  but  < 1^  value, 
highly  significant,  that  is  F > 1^  value. 
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Design  of  Experiments 


In  order  to  analyze  the  given  experimental  data  by  half -normal 
plotting,  we  shall  reduce  the  data  to  single  degree  of  freedom  sums  of 
squares.  The  method  to  be  used  depends  upon  the  definition  of  complete 
sets  of  orthogonal  contrasts  for  each  of  the  factors  A,  B,  and  C.  This 
definition  generally  is  somewhat  artibrary,  but  it  is  our  experience 
that  an  experimenter  familiar  with  the  nature  of  the  factor  levels  and 
the  purpose  of  the  experiment  can,  in  most  instances,  provide  suffi- 
cient justification  for  the  prior  definition  of  a meaningful  complete  set 
of  single  degree  of  freedom  orthogonal  contrasts  among  the  levels. 

The  use  of  orthogonal  polynomials  for  quantitative  levels  is  often  indi- 
cated, while  for  qualitative  levels,  meaningful  comparisons  among 
certain  levels  are  often  obvious.  On  occasion,  only  a partial  set  of 
orthogonal  comparisons  will  appear  to  be  of  intrinsic  value  and  it  may 
be  necessary  to  complete  the  orthogonal  set  by  adding  contrasts  of  no 
apparent  importance.  In  the  absence  of  useful  information  on  the  nature 
of  the  levels  (except  that  they  are  all  qualitative)  in  the  present  example, 
we  shall  be  totally  arbitrary  in  defining  the  contrasts,  but  will  attempt 
to  indicate  their  potential  interpretations.  -These  contrasts  are  shown 
in  Table  3.  For  factor  A,  contrast  Aq,  is  the  "null"  or  "average" 

contrast*,  while  A^  compares  the  average  of  levels  1 and  2 against 
the  average  of  levels  3,  4 and  5,  A^  compares  level  1 vs.  level  2,  A^ 
compares  level  3 against  the  average  of  levels  4 and  5 and  A^ 

compares  level  4 with  level  5.  The  orthogonality  of  the  set  is  evident 
in  that  the  coefficients  sum  to  zero  for  all  contrasts  except  the  null 
contrast  and  the  sum  of  products  of  coefficients  is  zero  for  all  pairs 
of  contrasts.  For  factor  B,  the  non -hull  contrasts  compare  level  1 
with  level  2 with  the  average  of  levels  1 and  3.  (In  another  context, 

Bj  and  B^  are  the  orthogonal  polynomials  for  three  equally  spaced 

levels,  Bj^  being  the  linear  contrast  and  B^  the  quadratic.  ) For  factor  C, 
the  contrasts  C^,  C2  and  Cj  make  the  following  comparisons  among  levels, 
respectively:  (1  and  2)  vs.  (3  and  4),  (1  and  3)  vs.  (2  and  4)  and  (1  and  4) 


❖ 

Daniel  1962  has  suggested  the  term  "null"  is  inappropriate  because 
of  the  generally  positive  expectation  of  this  contrast.  We  chose  the 
term  because  (i)  it  is  connoted  by  our  zero  subscript  notation,  (ii) 
this  contrast  is  not  a comparison  among  levels,  and  (iii)  this  contrast 
is  generally  "of  no  consequence"  in  the  analysis. 
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vs.  (2  and  3).  These  contrasts  would  be  of  interest,  e.  g.  , in  the  event 
that  factor  C incorporated  two  subfactors,  say  D and  E,  where 
levels  1 and  2 are  at  the  low  level  of  D and  levels  3 and  4 at  the 
high  level  of  D,  while  levels  1 and  3 are  at  the  low  level  of  E and  2 
and  4 at  the  high  level  of  E.  Then  Cj^  is  the  effect  of  D,  is  the 

effect  of  E and  is  the  interaction  of  D and  E. 

The  three  sets  of  contrasts  (A^,  A^,  Aj,  A^),  (B^,  B^)  and 
(Cl,  C^,  C^)  will  provide  a basis  for  reducing  the  sums  of  squares  for 
factor  A (4  d.  f. ),  factor  B (2  d.  f. ) and  factor  C (3  d.  f. ) to  indepen- 
dent single  degree  of  freedom  sums  of  squares.  It  remains  to  develop 
such  a basis  for  the  two-  and  three<-factor  interactions.  A natural 
method  for  accomplishing  this  is  the  extension  of  the  original  single 
factor  contrast  sets  to  interaction  contrast  sets.  This  method  is  exem- 
plified in  Table  4 for  Factors  B and  C.  All  possible  combinations 
of  the  levels  of  B and  C are  employed  as  columns,  while  rows  are 
contrasts.  For  any  combination  of  a particular  level,  say  i,  of  B 
with  a particular  level,  say  j,  of  C,  the  coefficient  in  the  contrast 
BqCr  is  obtained  by  multiplication  of  the  coefficient  of  level  i of  B 

in  the  contrast  B by  the  coefficient  of  level  j or  C in  the  contrast 
q 

C^.  Sums  of  squares  of  the  B and  C contrasts  may  be  obtained  by 
multiplication  of  the  corresponding  sums  of  squares  for  B and  for  C. 

Of  the  12  orthogonal  contrasts  in  Table  4,  is  the  null  contrast 

while  the  contrasts  B^Cj,  BqC2»  B0C3,  BjCq  and  B2CQ  are  simply  the 
original  contrasts  C^,  C3,  B^  and  B^,  respectively,  averaged  over 
all  levels  of  the  other  factor.  The  six  contrasts  BjC^,  B^C^,  BjC^, 

B2C2,  B2C3  are  new  and  constitute  a basis  for  partitioning  the  BC 

interaction  sum  of  squares  (6  d.  f.  ) into  orthogonal  single  degree  of 
freedom  sums  of  squares.  Application  of  this  method  will  likewise  pro- 
duce bases  for  partitioning  the  sums  of  squares  for  AB  (8  d.  f. ),  AC 
(12  d.  f. ) and  ABC  (24  d.  f.  ). 

The  above  method  of  defining  interaction  contrasts  is  incorporated 
in  the  method  we  employ  for  calculating  halfwnormal  variates  by  desk 


Table  4.  ORTHOGONAL  CONTRASTS  FOR  FACTORS  B AND  C 
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I calculator.  A sample  work  sheet  for  this  method  is  shown  in  Table  5a 

I and  Bb’!'.  Section  I of  this  table  is  merely  a recopying  of  the  original 

I data  from  Table  1.  In  this  section  a column  of  five  numbers  represents 

I the  observations  for  the  five  levels  of  A over  a particular  one  of  the 

I twelve  combinations  of  levels  of  B and  C represented  by  columns.  Section 

I II  is  computed  by  operating  on  these  columns  with  the  contrasts  Aq,  Aj^, 

I A2»  A3,  A4.  For  example,  products  of  the  coefficients  of  A^  with  the 

I corresponding  elements  of  a particular  column  are  formed  and  these 

j five  products  are  summed  and  entered  in  Section  II  in  the  first  row  of 

I that  column.  Similarly,  sums  of  products  of  coefficients  of  A j with 

I corresponding  elements  of  columns  are  entered  in  the  second  row  of 

I Section  II,  and  so  forth.  Thus  each  element  of  Section  II  is  formed  as 

a sum  of  products  of  coefficients  of,  say,  Ap  with  corresponding  ob- 
servations and  is  entered  on  the  ( pf  1 )-th  row  of  the  appropriate  column. 

[ Section  II  may  be  visualized  as  an  aggregation  of  20  rows  of  three 

! elements  each,  where  each  row  corresponds  to  a particular  contrast  of 

i A and  a particular  level  of  C.  The  three  elements  of  each  row  corres- 

! pond  to  the  three  levels  of  B.  Now  Section  III  is  formed  from  Section  II 

by  summing  products  of  coefficients  of  the  B contrasts  and  corresponding 
elements  of  each  row  of  three.  Each  such  sum  of  products  is  entered  in 
the  corresponding  row,  with  the  sum  of  products  from  coefficients  of  B^ 
entered  as  the  ( q + 1 )-th  element  of  that  row. 

i 

Section  III  may  be  visualized  as  comprising  four  sub-sections  of  15 
I elements  each,  with  the  elements  of  a sub- section  corresponding  to  a 

particular  contrast  of  AB  (i.  e.  , a particular  combination  of  a contrast 
of  A and  a contrast  of  B)  and  the  four  sub-sections  corresponding  to  the 
four  levels  of  C.  Section  IV  is  formed  from  Section  III  by  summing  pro- 
ducts of  coefficients  of  the  C contrasts  and  identically  placed  elements 
from  the  four  corresponding  sub-sections.  The  sum  of  products  is 
entered  in  the  corresponding  place  of  one  of  the  sub-sections  of  Section  IV, 
I with  the  sum  of  products  from  coefficients  of  entered  in  the  (r+l)-th 

i sub- section. 


* 

This  calculation  method  is  essentially  &e  same  as  that  given  in 
Appendix  8G  of  Davies  jl954,  pp.  363-6  I . In  some  instances  the 
format  in  Davies  (with  the  addition  of  final  column  of  half -normal 
variates)  may  be  preferred. 
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The  method  of  calculation  of  Sections  II,  III  and  IV  may  not  be  appar- 
ent at  first  glance,  but  verifying  part  or  all  of  the  data  in  Table  5a  from 
the  description  above  should  help  to  clarify  the  process.  Computing  clerks 
will  find  it  helpful  to  write  the  coefficients  of  each  contrast  on  a strip  of 
paper,  appropriately  oriented  vertically  or  horizontally  and  spaced  so 
that  when  overlaid  on  the  worksheet  each  coefficient  appears  adjacent 
to  the  element  to  be  multiplied. 

Section  V (Table  5b)  merely  identifies  the  elements  of  Section  IV  and 
subsequent  sections  according  to  the  contrasts  they  represent.  This 
identification  is,  of  course,  highly  systematic  and  might  well  be  omitted 
when  familiarity  with  the  method  is  attained. 

Section  VI  contains  the  "divisors'*,  obtained  by  multiplying  the  sums 
of  squares  of  the  coefficients  of  the  contrasts  A_,  B , C appropriate  to 

r ^ 

each  element,  as  found  in  Table  3. 

Section  VII  contains  the  single  degree  of  freedom  sums  of  squares 
corresponding  to  each  contrast.  Each  element  is  obtained  by  squaring 
an  element  of  Section  IV,  dividing  by  the  corresponding  element  of 
Section  VI  and  entering  in  the  corresponding  place  of  Section  VII. 

Section  VIII  contains  the  half-normal  variate  values,  each  of  which 
is  computed  as  the  square  root  of  the  corresponding  element  of  Section  VII, 
positive  or  negative  according  to  the  sign  of  the  corresponding  element  of 
Section  IV.  (It  would  perhaps  have  been  advisable  to  include  the  first 
decimal  of  each  of  these  values  in  order  to  discriminate  more  fully  among 
them. ) 

Certain  check  compiitations  in  the  method  have  been  omitted,  but  an 
over-all  check  can  be  readily  obtained  from  Section  VII  by  comparing 
sums  of  these  single  degree  of  freedom  sums  of  squares  with  the  usual 
analysis  of  variance  of  Table  2.  These  checks  are  indicated  in  Table  6. 

It  will  be  noted  that  all  sums  of  squares  agree  with  Table  2 within  the 
expected  rounding  error  accumulated  from  Section  VII. 

The  half-normal  variates  must  now  be  ordered  by  magnitude  before 
plotting.  This  ordering  is  shown  in  Table  7,  along  with  an  identification 
of  the  contrast  represented  (letters  with  subscripted  zeroes  have  been 
dropped)  and  the  appropriate  quantile  of  the  empirical  distribution,  de- 
fine d by 
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Table  6.  DEVELOPMENT  OP  USUAL  ANALYSIS  OP  VARIANCE 


Source 

d.f . 

Contrasts 

s.s. 

A 

4 

^1®0°0  ^2®0^0  ^3®0^0 

^4®0®0 

478462 

B 

2 

^O^l^O  ^0^2^0 

52795 

C 

3 

^0®0®1  ^0^0®2  ^0^0®3 

150239 

A£ 

6 

^3®1°0 

^4®1^0 

^1®2^0  ^2®2®0  ^3®2°0 

A4B2C0 

16808 

AC 

12 

AiBqCi  AgB^C^  ^3®0^1 

^4®0®I 

^1®0®2  ^2®0^2  ^3®0®2 

^4®0®2 

^1®0°3  ^2®0®3  ^3®0®3 

^4®0^3 

53890 

BC 

6 

^0®1^1  ^0®1®2  ^0®1°3 

^0®2°1  ^0®2°2  ^0®2®3 

6417 

ABC 

24 

AiB^Ci  AgB^Cj^  ^3®1^1 

a^b^c^ 

^1®1°2  ^2®l°2  ^3®i°2 

%V2 

^1®1°3  ^2®1°3  ^3®1®3 

A4B1C3 

A1B2C1  AgBgC^  A3B2C^ 

^4®2°1 

^1®2°2  ^2®2°2  ^3®2®2 

^4®2®2 

'^1®2®3  ^2®2^  ^3®2°3 

A/^BgC^ 

-7690 

Total 

59 

766301 

Mean 

1 

^0®0°0 

6069348 

Raw  total 

60 

6835649 

Table  7.  HALF-NORMAL  VARIATES 


Order 

k 

Variate 

Contrast 

Quantile 

Order  Variate 
k 

Contrast 

Quantile 

^k 

60 

2464 

Null  • 

i 

30 

24 

AoB^Co 

.5000 

59 

447 

^3 

.9915 

29 

23 

^4®1^1 

.4831 

58 

395 

^1 

.9746 

28 

22 

^2^2 

.4661 

57 

367 

“3 

.9576 

27 

20 

-Vi 

.4492 

56 

294 

*2 

.9407 

26 

18 

-AgBjCi 

.4322 

55 

230 

.9237 

25 

17 

*3®2®3 

.4153 

54 

191 

.9068 

24 

17 

-A4C2 

.3983 

53 

152 

-AgOj 

. 8898 

23 

17 

A2B1C2 

.3814 

52 

132 

A^Cs 

.8729 

22 

16 

""'^3^2  ^1 

.3644 

51 

111 

°2 

.8559 

21 

15 

^1®2 

.3475 

50 

94 

.8390 

20 

14 

B,Ci 

.3305 

49 

64 

*3®1 

.8220 

19 

13 

A2B2C1 

.3136 

48 

59 

AjCg 

.8051 

18 

12 

ApBgC. 

.2966 

47 

54 

-=1 

.7881 

17 

11 

-A^B^^C^ 

.2797 

46 

50 

-A3Bg 

.7712 

16 

9 

-AgBpCg 

.2627 

45 

50 

-BgCj 

.7542 

15 

9 

"®2 

.2458 

44 

47 

-B2'=2 

.7373 

14 

8 

-A  B C3 

.2288 

43 

46 

.7203 

13 

7 

.2119 

42 

46 

-A3O, 

.7034 

12 

7 

~^3®2^2 

. 1949 

41 

■ 42 

-A^C, 

.6864 

11 

6 

-B1C3 

.1780 

40 

38 

A^Og 

.6695 

10 

5 

A^B^Cg 

.1610 

39 

36 

.6525 

9 

4 

^1^1°2 

.1441 

39 

34 

AjjBg 

.6356 

8 

4 

“■^1®2^1 

.1271 

37 

33 

■*3°3 

.6186 

7 

3 

-AgBg 

.1102 

36 

32 

*1®2°3 

.6017 

6 

3 

^1®2°2 

.0932 

35 

29 

■*4®2°3 

.5847 

5 

2 

-AiBi 

.0763 

34 

29 

*3®1°2 

.5678 

4 

1 

-AiBiCi 

.0593 

33 

27 

-Vl°l 

.5508 

3 

1 

AhBpCp 

.0424 

32 

27 

-B,Oi 

.5339 

2 

0 

-A4B1C2 

.0254 

31 

27 

%°2 

.5169 

1 

0 

.0085 
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where  k is  the  rank  order  and  n is  the  number  of  variates.  Here,  as 
in  most  instances,  it  seems  appropriate  that  the  null  contrast  be  excluded 
from  the  variates  to  be  examined.  The  sign  of  the  contrast  is  now 
attached  to  the  label  and  only  positive  variates  are  plotted. 

The  variate  values  and  quantiles  are  next  plotted  on  half^-normal 
probability  paper  (as  in  Figure  1)  for  interpretation.  Discussion  of  the 
interpretation  phase  of  the  analysis  of  this  example  will  be  deferred  to 
a later  section. 

3.  SOME  THEORY*.  At  this  point  we  shall  touch  briefly  on  some 
theoretical  aspects  of  the  development  of  half-normal  variates  from  multi- 
level factorial  experiments.  To  simplify  the  discussion  we  shall  assume 
that  we  are  concerned  with  a threej-factor  experiment,  although  it  should 
be  remembered  that  the  theory  and  methodology  apply  with  equal  validity 
to  any  number  of  factors. 

We  denote  by  the  observation  obtained  with  factor  A at  level  h, 

factor  B at  level  i and  factor  C at  level  j,  where  h = 1, 2, . . . , a; 
i = 1,  2,  . . . , b;  j = 1,  2,  . . . , c.  The  coefficients  of  the  orthogonal 
contrasts  for  factor  A will  be  indicated  by  apj^,  denoting  the  coefficient 

for  level  h in  the  p-th  contrast.  Similarly  the  coefficients  of  the  con- 


trasts for  factors  B and 


are  denoted 


respectively. 


We  assume  that  for  each  factor  there  is  a null  contrast,  these  being  de> 
noted  Aq,  Bq,  and  defined  by 


“oh  = ‘’oi  = “oj  = *=  ““ 


This  section  is  based  on  well-known  results  concerning  distributions  of 
linear  functions  of  random  variables  and  may  be  verified  by  reference 
to  standard  introductory  texts  on  mathematical  and  theoretical  statistics. 
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Furthermore,  by  the  definition  of  orthogonal  contrasts, 


^ ph  qi 

h ^ i ^ 


y b . = y c . = 0 
^ qi  L.  rj  ^ 


and 


P^fj^*****  ^“1,  b-lj  r = l,2,a>.,  r — lj 


^ ^ph  ^p'h 
h 1 


? ‘^qi  ^q'i  “ ? ^rj  ‘^r'j 


p ^ p';  q ^ q‘;  r ^ r'. 
The  three -factor  contrasts  are  defined  by 


(ApB^Cja)  = Z E E b^.  c^.  yj^. 


P'-Ojlj***])  3."lj  b“lj  r — 


Suppose  that  there  are  no  treatment  effects  , i.  e.  , 


E {y,  .}  = K';  all  h,  i,  j; 

hij 


and  that  the  experimental  errors  are  independent  and  have  constant 
variance  for  all  observations,  i.  e.  , 


^ (<yhij  - j- 


The  symbol  E C3  denotes  the  mathematical  expectation  operator. 
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Then 


unless 


Furthermore  , 


E (a^B  cj  = 0, 
p 4 


P = 0,  q = 0 and  r = 0,  in  which  case 


E {Aq  BoCj=  abc 

' ( E ai)(  E E <=!,)  0•^ 


h P"  i 


Denote  by  Ypqj.  the  variate  defined  by 


’fpqr  = <ApBqC,) 


yVlEa^,)  (E^MEc^j). 


Then 


e(y  } = V"ab^  U; 

000  ' 

E = 0,  unless  p=0,  q = 0,  r = 0; 


''  = O'  • 


If  the  experimental  errors  are  normally  distributed,  then  the  are 

normally  distributed.  (Under  fairly  weak  assumptions  the  Yp^^.  will  tend 

to  be  normally  distributed  in  large  experiments  even  for  non-normal 
distributions  of  experimental  error. ) Then  the  non-negative  half-normal 
variates, 

The  symbol  V [ j denotes  the  variance  operator,  V {.xj  = E ((X-E(x})^}  . 
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X 


pqr 


pqr 


c/nE  a|j,)  ( E *■^1  ( T.  cljh  p.  q.  0.  0,  0 

h i ^ j 

are  indeed  distributed  according  to  the  half-normal  density 

f(x)  = Vz/rr  exp(-X^/2cr^).  X>0 

X<  0. 


From  this  result,  the  half-normal  variates  for  multi-level  experiments 
may  be  seen  to  be  essentially  equivalent  to  those  for  2^  experiments, 
making  the  work  of  Daniel  1959  and  Birnbaum  1959  relevant  to  the 
interpretation. 

4.  INTERPRETATION  OF  EXAMPLE.  We  shall  turn  now  to  the 
interpretation  of  the  example  given  earlier.  Some  difficulty  will  be 
experienced  because  of  our  ignorance  of  the  precise  nature  of  the  factors 
and  their  levels,  the  experimental  techniques  and  the  observations  them- 
selves, but  we  shall  attempt  to  proceed  along  lines  suggested  by  Daniel 
for  2^  experiments. 

To  recapitulate  the  results  of  Section  2,  we  have,  in  Table  7,  59 
ordered  variates  — Xpqj.  whose  empirical  cumulative  distribution 

should  resemble  the  cumulative  half-normal  distribution  under  the 
hypothesis  that  there  are  no  treatment  effects.  We  have  plotted  these 
values  against  their  quantiles  in  Figure  1,  where  they  should  be  approxi- 
mately linear  under  the  null  hypothesis. 

We  note  at  a glance  that  the  plotted  points  are  markedly  and  systemati- 
cally non-linear.  In  fact,  a little  preliminary  geometrical  construction 
leads  us  to  believe  that  a number  of  the  variates  are  too  large  to  have 
arisen  by  chance  under  the  null  hypothesis.  The  rationale  for  this  belief 
is  as  follows.  Under  the  null  hypothesis  the  standard  deviation,  ^ , is 
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directly  approximated  by  the  value  of  X where 

m 


m = (0.  683  n + 0.  5), 


= 41,  approximately. 


From  Table  7, 


= 42. 


Then,  under  the  null  hypothesis,  the  plotted  points  should  Ub  near  a 
straight  line  through  the  origin  and  the  point  indicated  in 

Figure  1.  Should  the  largest  X He  "far  enough"  to  the  right  of  this  line 
it  is  reasonable  to  presume  that  it  did  not  arise  by  chance  under  the  null 
hypothesis.  It  may  then  be  taken  as  real  and  the  next  largest  X promoted 
to  the  largest.  This  is  roughly  equivalent  to  increasing  the  ordinate  bf 
the  second  point  to  that  of  the  first  point.  Should  this  replotted  point  also 
lie  "far  enough"  to  the  right  of  the  line,  it  too  may  be  judged  real  and 
excluded,  promoting  the  next  X to  the  largest,  etc.  In  Figure  1,  we  make 
a crude  test  of  the  largest  values  by  constructing  a horizontal  through 
the  largest  point  to  intersect  the  previously  constructed  empirical  cu- 
mulative distribution  line.  From  this  intersection  we  drop  a vertical 
line  and  observe  that  all  contrasts  represented  by  points  lying  to  the  right 
of  this  vertical  would  have  to  be  excluded  before  the  largest  X would  He  on 
or  above  the  original  c.  d.  line.  In  this  crude  manner  we  judge  from 
Figure  1 that  six  to  ten  of  the  largest  values  of  X would  be  unlikely  to 
occur  under  the  null  hypothesis.  This  graphical  construction  is  no 
"exact"  test;  in  fact  it  is  rather  likely  that  one  or  more  contrasts  would 
be  judged  "real"  in  this  manner  even  if  the  null  hypothesis  did,  in  fact, 
hold.  There  is  one  element  of  conservativism  in  this  procedure,  in  that 
the  plotted  c.  d.  Hne  is  based  upon  all  contrasts,  while  a c.  d.  Hne  based 
only  on  contrasts  not  judged  "real"  at  this  stage  would  He  to  the  left  of 
the  original  line. 

Let  us  tentatively  suppose  that  the  six  largest  contrasts  (A3,  Aj, 

C3,  A2»  A^)  are  real,  considering  (after  Daniel  [i959j  p.  315^  ) 

their  simple  names,  as  well  as  their  magnitudes  relative  to  the  rest  of 
the  set.  We  plot  anew  the  53  remaining  contrasts  in  Figure  2.  Actually, 
in  addition  to  the  ten  largest  remaining  contrasts,  only  a fraction  of  the 
points  are  plotted,  together  with  the  c.  d.  Hne  through  (X^,  where 
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m = (0.  683)  (53)  + 0.  5, 
= 37,  approximately. 


The  values  of  are,  of  course,  recalculated  for  n = 53.  It  appears 
reasonable  to  judge  from  this  plot  that  the  four  largest  contrasts 
^2^3*  ^2*  ^2^1^  are  real. 

A final  plot  of  the  values  obtained  after  eliminating  the  ten  largest 
values  is  shown  in  Figure  3.  It  appears  in  tiiis  plot  that  all  real  effects 
have  been  removed,  with  a residual  error  standard  deviation  approximately 
equal  to 


Ka  = 841 
34 


(The  actual  mean  square  of  the  49  residual  contrasts  is  8l6, ) 

Some  further  details  of  interpretation  might  be  attempted.  For  ex- 
ample, there  is  a suggestion  in  Figure  1 and  in  Table  7 that  there  may 
have  been  plot- splitting,  with  factor  B applied  within  plots.  This  also 
appears  plausible  from  the  rudimentary  information  given  as  to  the 
nature  of  this  factor.  A further  plotting,  not  ahown  here,  in  which  con- 
trasts including  or  B^  were  separated  from  those  containing  B^ 
suggests  a whole  plot  standard  deviation  of  about  50-60  and  a split-plot 
standard  deviation  of  about  20-25. 


5.  COMPUTER  USE.  We  have  used  half-normal  plots  for  multi- 
level factorial  experiments  for  almost  two  years.  Our  first  major  attempt 
to  employ  this  technique  was  in  the  analysis  of  an  unreplicated  10  x 5 x 
3x2^  experiment.  The  factor  levels  in  this  experiment  were  applied 
in  a split-split-split  plot  design  and  certain  problems  of  variance  hetero- 
geneity were  apparent.  The  half-normal  plotting  of  this  data  was  suffi- 
ciently informative  that  it  appeared  worthwhile  to  develop  a program  for 
the  IBM  1620  to  be  employed  in  computing  half -normal  variates  from 
multi-level  factorial  data.  This  program.  Single  Degree  of  Freedom 
Analysis  of  Variance  (SIDOF),  has  a capacity  of  eight  factors,  each  at 
two  to  ten  levels.  It  requires  as  input  the  observations  and  normalized 

vectors  of  contrast  coefficients  CX.  , P,  , etc.  , where 

o a r 


FIGURES.  HALF- NORMAL  PLOT,  TEN  LARGEST  CONTRASTS  OMITTED 
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Each  factor  requires  an  additional  "pass"  through  the  machine.  On  the 
first  pass,  the  machine  computes  the  quantities  (assuming  three  factors), 


(Ap)y  = Z O'ph  Vhij- 

h 

On  the  second  pass  are  computed  the  quantities. 


(A  B ).  = 
P q J 


L 5 . (A  ).. 

Y ^qx  p'lj 


and  on  the  third  pass  the  quantities 


<ApB,C^)  = E 

j 


i . (A  BJ.. 
rj  P q J 


At  each  pass  the  output  includes  both  the  (signed)  contrasts  developed  and 
their  squares.  This  program  was  one  of  the  first  developed  for  the  IBM 
1620  at  Dugway  Proving  Ground  and  consequently  was  employed  for  a 
short  period  of  time  as  a general-purpose  analysis  of  variance.  (It  is, 
of  course,  much  slower  than  other  general-purpose  programs  available.  ) 
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6.  EXPERIENCE.  Some  general  comments  on  our  experiences  with 
half-normal  plots  for  multi-level  factorials  may  be  in  order.  We  shall 
be  guided  in  this  commentary  largely  by  the  approach  of  Daniel  Q959J  . 

a.  Graph  Sheets.  We  have  generally  used  half-sheets  of  the  Prob- 
ability Scale  X 90  Divisions  paper  available  from  Keuffel  and  Esser 
(Nos.  358-23  and  359-23).*  Similar  papers  are  available  from  several 
other  sources.  These  papers  are  not  particularly  well-suited  to  the  pur- 
pose. It  would  appear  that  special  half-normal  paper  might  be  commer- 
cially feasible,  but  it  is  not,  to  our  knowledge,  currently  available. 

b.  Birnbaum's  test  statistic.  The  test  statistic  developed  by 
Allan  Birnbaum [ I959J  has  been  used  for  our  purposes.  Birnbaum’s 
work  has  been  particularly  oriented  toward  2^  experiments  and  studies 
of  the  behavior  of  this  statistic  in  multi  -level  factorials  vo  uld  be  useful. 


c.  Defective  values.  Daniel  indicates  the  utility  of  half-normal 
plotting  in  ZP  experiments  for  detecting  defective  values.  For  multi- 
level factorials  the  presence  of  defective  values  appears  more  difficult 
to  diagnose,  particularly  with  unrestricted  sets  of  orthogonal  contrasts. 
The  isolation  of  the  particular  defective  values  is  also  more  difficult. 

Plot  “Splitting.  The  effect  of  plot -splitting  upon  the  halfj?normal 
plots  for  multi-level  experiments  is  similar  to  that  described  by  Daniel. 
We  have  some  reservations  concerning  indiscriminate  searches  for  plot- 
splitting, however.  It  is  generally  accepted  that  in  most  experiments 
two-factor  interactions  tend  to  be  smaller  than  main  effects,  three - 
factor  interactions  tend  to  be  smaller  than  two-factor  interactions,  etc. 
(Here  we  are  speaking  of  real  effects  and  interactions,  though  perhaps 
of  negligible  magnitude.  ) Thus  in  actual  experiments  the  slope  of  half- 
normal plots  may  be  expected  to  increase  with  the  relative  number  of 
high  order  interactions  included.  The  plotted  results  of  an  experiment 
involving  a number  of  small  but  real  interactions  may  appear  very  simi- 
lar to  the  results  induced  by  plot-splitting,  since  split  plot  error  con- 
trasts invariably  contain  a relatively  larger  number  of  the  higher  order 


The  graph  sheets  used  in  Figures  1,  2,  and  3 were  reproduced  from 
a master  kindly  provided  by  Mr.  Daniel.  It  is  hoped  that  such  sheets 
will  soon  be  published. 
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contrasts.  Our  practice  is  generally  to  employ  a split  plot  analysis 
only  when  knowledge  of  the  experimental  techniques  indicates  its  pro- 
priety. 

e.  Convexity  of  plots.  The  detection  of  antilognormal  distribution 
of  error  by  downward  convexity  of  half-normal  plots  appears  difficult, 
as  indicated  by  Daniel  [1959,  p.  336  J.  Most  of  our  analysis  work  is, 
however,  based  on  transformed  data  and  we  have  seldom  experienced 
this  particular  anomaly.  In  any  event,  the  averaging  effect  of  the  con- 
trasts would  presumably  minimize  the  effects  of  non-normality  of  error. 
On  the  other  hand,  we  have  noted  that  the  removal  of  a moderate  number 
of  points  representing  apparently  real  effects  often  results  in  a downward 
convexity  of  the  ypper  portion  of  th<e  plot.  We  generally  attribute  this 
appearance  to  the  inadvertent  removal  of  one  or  more  points  atepresenting 
error  contrasts,  for  the  result  looks  very  much  like  the  plot  of  a normal 
distribution  with  truncated  upper  tail. 
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